Image registration brings a floating image into spatial alignment with a reference image. This enhances image correlation, removes geometric distortion, and facilitates various image processing tasks, such as image analysis, segmentation, understanding, visualization and rendering. Image registration has wide applications in medical imaging, video motion analysis, remote sensing, security and surveillance. For example, in medical imaging using a computer aided diagnosis (CAD) system, abnormalities can be detected by frame subtraction of the same body parts after registration. Without proper image registration, there is an ambiguity between the abnormalities and the changes due to geometric distortion. In a picture archive and communication system (PACS), automatic registration of images captured at different times can enhance visualization and image rendering and help a radiologist quickly identify image changes for accurate and efficient diagnosis.
Fully automatic image registration has been successfully used in selected applications, when the images from the same modality have similar appearance and the transform involves only translation and rotation. Other cases have turned out to be quite challenging, especially when the imaging sources and the underlying motion have been unconstrained.
In multimodal image processing, images of the same body parts are captured from different modalities (for example, X-ray mammography and MRI breast imaging), which potentially can improve system performance in terms of accuracy, robustness and speed from the complementary information. Various multimodal image registration methods have been proposed. “A survey of medical image registration” by J. B. A. Maintz and M. A. Viergever, Medical Image Analysis, vol. 2(1), pp. 1-36, (1998); and “Mutual-information-based registration of medical images: a review” by J. P. W. Pluim, et al., IEEE Transaction on Medical Imaging, Vol. 22, pp. 986-1004, August 2003, provide comprehensive reviews.
The use of histogram-based mutual information (HMI) for multimodal image registration is disclosed in: “Alignment by maximization of mutual information” P. Viola and W. M Wells III, International Journal on Computer Vision, vol. 24(2), pp. 137-154, 1997; “Multi-modal volume registration by maximization of mutual information” W. M. Wells III, et al., Medical Image Analysis, vol. 1(1), pp. 35-51, 1996; and “Multimodality image registration by maximization of mutual information” F. Maes, et al., IEEE Transactions on Medical Imaging, vol. 16, pp. 187-198, April 1997.
Most histogram-based mutual information (HMI) methods have the shortcoming of building statistics upon an image intensity histogram and ignore spatial variations, which adversely impacts the registration accuracy and robustness. An exception which uses high order statistics is disclosed in “Non-rigid registration using higher-order mutual information” D. Rueckert, et al., Proceedings of the SPIE Conference on Medical Imaging, Vol. 3979, pages 438-447, (2000). This approach has the shortcoming that a compromise is made to balance the quantization levels and the number of bins (equivalent to balancing space and precision), especially when image comparison is performed regularly, such as in iterative image registration applications.
U.S. Pat. No. 6,563,941, “Model-based registration of cardiac CTA and MR acquisitions”, to T. O'Donnell, et al., and U.S. Published Patent Application US2003/0216631, “Registration of thoracic and abdominal imaging modalities”, by I. Bloch, et al., disclose an approach, in which a single feature is used for image representation in the registration process. This limits accuracy and robustness.
Mutual information (MI) analysis measures the dependency of two random distributions based on the Kullback-Leibler divergence. Given a random variable X, the probability density function p(X) measures the frequency of X having a particular value. The entropy of X,H(x)=−Σp(x)log p(x),indicates its randomness. Small entropy means X is strongly biased to certain events and an entropy of zero means an event is certain. With two or more random variables X and Y, the joint probability density function p(X,Y) measures the frequency that X and Y have particular values, and p(X) and p(Y) are referred to as the marginal density functions. Similarly, the joint entropy is defined asH(x,y)=−Σp(x,y)log p(x,y)and the mutual information can be written asI(x,y)=H(x)+H(y)−H(x,y)
It would thus be desirable to provide methods of image registration capable of handling complex images from different modalities, which use multiple image features.